Resynchronization of a target volume with a source volume

ABSTRACT

An improved method and apparatus for quickly and efficiently updating the original source volume and original target volumes after the original source volume has become temporarily unavailable. The original target volume is characterized as a source volume while the original source volume is temporarily unavailable. Transfer lists of different data blocks are generated. Data blocks not originally found on a source are copied to the target. Data blocks included on a target that were not found on the source are removed. By focusing upon specific data blocks, this technique avoids the use of filer overhead and other computational resources that would be expended if the entire volume were recopied.

CROSS-REFERENCE TO RELATED APPLICATION

This application hereby incorporates by reference and is a continuationof application Ser. No. 09/825,855, filed Apr. 3, 2001, now U.S. Pat.No. 6,668,264.

BACKGROUND OF THE INVENTION

1. Field of the Invention

This invention relates to data storage systems; in particular, theinvention relates to synchronization of source and target volumes in amirrored storage system.

2. Related Art

Snapshots and multiple volumes are frequently used to prevent data losswhen a data storage drive fails in a file system. Such snapshots“capture” the contents of the files and directories in a volume at aparticular point in time in order to recover earlier versions of a filefollowing an unintended deletion or modification. Such snapshots canalso be copied to one or more volumes, which then can be used as amirror or a collection or mirrors and which can provide a back-up copyof the file system. When used in this way, the mirror can be referred toas a target volume. In general, a target volume is a “read-only” volumethat contains a set of data that is equivalent to the set of data on anoriginal source volume. Such target volumes can be written to only bythe original source volume.

A target volume may be updated periodically with respect to a sourcevolume by looking to the most recent snapshot that the target and sourcehave in common and using that snapshot as a consistency point (CP). Thefile blocks in the most recent common snapshot and the file blocks of anew snapshot are compared. The set of differences resulting from thiscomparison are written to the less up-to-date volume. In this way, bothsource and target volumes maintain equivalent sets of file blocks.

A source volume may become unavailable due to a failure of the sourcevolume or to a failed connection to the source volume. Under suchconditions, it is advantageous to temporarily use the target volume as asource volume by designating it as a “read/write” volume. User I/Os aredirected to write to the target volume while the original source volumeis unavailable.

One problem with writing to a target volume is that it may cause thetarget volume to contain data not found in the original source volume. Apartial solution to this problem involves transferring data from thetarget to the source once the source is restored. However this isundesirable because it requires diversion of computational resources andfiler overhead.

Accordingly, it would be advantageous to provide an improved techniquefor quickly and efficiently updating source and target volumes after atarget volume has been written to. This is achieved in an embodiment ofthe invention that addresses the foregoing deficiencies.

SUMMARY OF THE INVENTION

The invention provides an improved method and apparatus for quickly andefficiently updating an original source volume and an original targetvolume after the original target volume has been used as a source volumein a file system. One or more snapshots are used to compare dataincluded in the source and target volume. Instead of transferring theentire volume, only the data that is missing from a source and a targetvolume is transferred.

In a first aspect of the invention, a target volume becomes synchronizedwith a source volume after the target has been written to by an entityother than the source. Synchronization is a two phase process. In thefirst phase, a target volume provides the source volume with a list ofsnapshots and associated snapshot numbers used to determine the sequenceof the snapshots on the list. The source volume compares its own list ofsnapshot numbers with the list of the target volume's snapshot numbersand determines the newest common snapshot. This newest common snapshotis a consistency point between the target volume and the source volume.The source volume sends the target volume a set of snapshot numbersdescribing the newest common snapshot and the target volume reverts backto this snapshot. In the second phase, file system software identifiesall the data blocks contained in any one or more of the snapshots of thesource volume. This file system software also identifies all the datablocks in any one or more of the snapshots of the target volume usingthe data sent by the target volume to the source volume as describedsupra. A set of data blocks that are included in the source volume andnot included in the target volume is generated. This can be accomplishedby making a comparison based on logical differences, generating avirtual or actual list or other techniques known in the art.

File system software synchronizes the target volume with the sourcevolume. First, the file system software removes snapshots from a targetvolume if the snapshots are not included in the source volume's snapshotlist. Second, the file system software adds the set of data blocksidentified above (that is the set of data blocks that are included inthe source volume and not included in the target volume) to its memory.Lastly, the file system software adds snapshots to the target volume ifthe snapshots are included in the source volume's snapshot list and notin the target volume's snapshot list. At this point, the target volumeincludes the data blocks that are present on the source volume.

In a second aspect of the invention, the roles of the target volume andsource volume are reversed and the process described supra is performedagain so as to synchronize source volume with the target volume. This isnecessary because the target volume may include data blocks not includedin the source volume. After both source and target volumes aresynchronized, the target volume stops being written to and the sourceonce again is used as the active file. This is accomplished by 1)designating the target as a read-only volume, 2) designating the sourceas a read/write volume, and 3) redirecting users' I/O's back to thesource volume.

In a preferred embodiment, sources and volumes can be synchronizeddynamically, using a WAFL (Write Anywhere File Layout) system using RAID(Redundant Arrays of Independent Disks) architecture. However, variousother types of file systems involving redundant copies of data can alsobe used.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 shows a block diagram of a system for synchronizing a targetvolume to a source volume.

FIG. 2 shows a flow diagram of a method for synchronizing a targetvolume to a source volume.

FIG. 3 shows a flow diagram a method for synchronizing a target volumeand a source volume to each other.

Lexicography

The following terms are related to aspects of the invention as describedbelow. The general meanings of these terms are exemplary and in no waylimiting.

Source volume—in general, the term “source volume” refers to aread/write volume.

Target volume—in general, the term “target volume” refers to a read-onlyvolume that is used to back-up other data. However, in the event that asource volume becomes unavailable, a target volume may be designated as“read/write” and used as a source.

Synchronize—in general, the term “synchronize” refers to the process ofconforming a first set of snapshots to a second set of snapshots.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT

In the following description, a preferred embodiment of the invention isdescribed with regard to preferred process steps and data structures.Embodiments of the invention can be implemented using general-purposeprocessors or special purpose processors operating under programcontrol, or other circuits adapted to particular process steps and datastructures described herein. Implementation of the process steps andstructures described herein would not require undue experimentation orfurther invention.

System Elements

FIG. 1 shows a block diagram of a system for synchronizing a targetvolume to a source volume.

A system for synchronizing a target volume to a source volume (shown bygeneral character reference 100) includes a file system 110, upon whichresides one or more source volumes 120, one or more target volumes 140,and logic 115. In a preferred embodiment, the file system 110 is part ofa larger computer system including a memory and a processor. The filesystem 110 is coupled to an I/O port 105.

The source volume 120 includes a set of data blocks 125 and a set ofsnapshots 130. The set of data blocks 125 both data and meta-data.

The set of snapshots 130 includes individual snapshots that correspondto the set of data blocks 125 at various points in time. A snapshotincludes a map of blocks at a consistent point in the file system, butpreferably not the blocks themselves. The individual snapshots includesnapshot numbers which refer to the relative age of the snapshot. In apreferred embodiment, the higher snapshot numbers correspond to morerecent snapshots and lower snapshot numbers correspond to oldersnapshots. Although the snapshot numbers shown in FIG. 1 aresequentially numbered from one to six, the numbering of the snapshotsmay reflect deletion of a particular snapshot. Both the number and typeof data blocks in the set of data blocks 125, as well as the number andsequence of snapshots in the set of snapshots 130, are exemplary and inno way limiting.

In a preferred embodiment, the source volume 120 is a read/write volumethat receives user I/Os 105. A system 100 may include a plurality ofsource volumes 120. In the event that a source volume 120 becomesunavailable, a target volume 140 may be temporarily used as a sourcevolume 120.

The target volume 140 includes a set of data blocks 145 and a set ofsnapshots 150. Similar to the set of data blocks 125 included in thesource volume 120, the set of data blocks 145 includes individual datablocks, indirect data blocks, and double indirect data blocks. The setof snapshots 130 (which are themselves data blocks) includes individualsnapshots of the set of data blocks 145 at various points in time. Theindividual snapshots include snapshot numbers relating to the relativeage of a snapshot Generally, the highest snapshot number corresponds tothe most recent snapshot. Similar to the snapshot numbers included inthe target volume 120, these snapshot numbers need not be a uniformsequence. Both the number and type of data blocks in the set of datablocks 145, as well as the number and sequence of snapshots in the setof snapshots 150, are exemplary and in no way limiting.

In a preferred embodiment, the target volume 140 is a read-only volumethat is preferably used to replicate data from the source volume 120.When used as such, user I/Os 105 are not directed to the target volume140, but rather to the source volume 120. A system 100 may include aplurality number of source volumes 120 and target volumes 140, such thatthe source volumes 120 mirror data to the redundant target volumes 140.

The logic 115 provides a technique for synchronizing a source volume 120to a target volume 140, a target volume 140 to a source volume 120, orboth. Generally, this technique is used when a source volume 120 istaken off line and a target volume 140 is temporarily used in it's placeor when the target volume 140 is written to by any entity s other thanthe source volume 120.

Method of Use

FIG. 2 shows a flow diagram of a method for synchronizing a targetvolume to a source volume.

A method for synchronizing a target volume to a source volume (shown bygeneral character reference 200) is performed by a system 100. Althougha method 200 is described serially, steps of a method 200 can beperformed by separate elements in conjunction or in parallel, whetherasynchronously, in a pipelined manner, or otherwise. There is noparticular requirement that a method 200 be performed in the same orderin which this description lists the steps, except where so indicated.

At a flow point 205, a system 100 is ready to perform a method 200 andsynchronize a target volume 140 to a source volume 120. The method 200is preferably performed after a target volume 140 has been madewritable.

In a step 210, the logic 115 identifies a set of snapshots 150associated with a target volume 140 and copies the identifiersassociated with those snapshots 150 to a source volume 120. For example,if the set of snapshots 150 includes snapshot numbers 1, 3, 4 and 6,those particular snapshots numbers are copied to the source volume 120.

At a step 220, the logic 115 compares the identifiers associated withthe set of snapshots 150 identified in step 210 with the set ofsnapshots 130. The most recent snapshot that is common to both sets isselected. For example, if the identifiers include snapshot numbers 1, 3,4 and 6 and the set of snapshots 130 includes snapshot numbers 1, 2, 4,and 5, then the most recent snapshot common to both sets is snapshotnumber 4.

At a step 225, the identifier associated with the most recent commonsnapshot (that is, the snapshot selected in step 220) is copied from thesource volume 120 to the target volume 140. During this step, an alertmay be sounded, informing the user I/O 105 that some of the data blocksunique to the snapshots newer than the newest common snapshot may belost and a prompt for “confirmation” or “abort” may be issued. If thereis a confirmation, the target volume 140 reverts back to the most recentsnapshot selected in step 220. This reversion may be referred to as a“SnapRestore”.

At a step 228, the source volume 120 generates a new snapshot of itself.This snapshot is used to preserve the set of data blocks 125 at thesource volume 120 and to determine the incremental transfer of datablocks between the source volume 120 and target volume 140.Simultaneously, the target volume 140 is designated as a read onlyvolume.

In a preferred embodiment, the source volume 120 may continue receivingreads and writes from clients.

At a step 229, the snapshots that are not included in the source volume120 are removed from the set of snapshot 150 on the target volume 140.In this way, the target volume becomes synchronized with respect tosnapshots that are not present on the source volume 120.

At a step 230, the union of the data blocks in the set of snapshots 130is computed. This union will preferably include available and allocateddata blocks from any one or more target snapshots 130. This step ispreferably performed by the source volume 120. Steps 235 and 245 occursimultaneously with step 230.

At a step 235, the union of the blocks in the set of snapshots 150 iscomputed. This union will preferably include available and allocateddata blocks from any one or more target snapshots 150. This step ispreferably performed by the source volume 120. Step 235 is performed atthe same time as steps 230 and 245.

At a step 245, difference between the unions calculated in steps 230 and235 is calculated. This difference represents the blocks to betransferred. These blocks are copied from the source volume 120 to thetarget volume 140. In this way, the target volume 140 becomessynchronized with respect to blocks present on the source volume 120.

At a flow point 260, the target volume 140 is synchronized with respectto the source volume 120 and the method 200 is complete.

FIG. 3 shows a flow diagram for synchronizing a target volume and asource volume to each other.

A method for aligning a target volume and a source volume to each other(shown by general character reference 300) is performed by a system 100.Although the method 300 is described serially, the steps of method 300can be performed by separate elements in conjunction or in parallel,whether asynchronously, in a pipelined manner, or otherwise. There is noparticular requirement that method 300 be performed in the same order,in which this description lists the steps, except where so indicated.

At a flow point 310, system 100 is ready to perform a method 300. Themethod 300 compensates for the asynchronous nature of mirroring data. Atthis time, both the source volume and target volume are taken off line.This prevents users form writing to either volume during the transition.

At a step 315, the target volume 120 becomes re-synchronized with thesource volume 140 as shown in FIG. 2. Upon completion of step 315, thetarget volume 120 includes information stored on the source volume 140.However, given the asynchronous nature of mirroring from the sourcevolume 140 to the target volume 120, the source volume 140 may include alittle bit of data that is not present on the target volume 120. Thisinconsistency is rectified in the following steps.

At a step 320, the target volume 120 is designated as read/write.However, user I/Os are not directed to it at this time.

At a step 330, the method 200 is performed again such that the roles ofthe target volume and source volume are reversed so that data on thesource volume 140 is mirrored to the target volume 120. Upon completionof this step, the target volume includes all of the date that wasoriginally on the source volume 140 and does not include any data thatwas not on the source volume 140.

At a step 335, the target volume 120 is designated as a read only andthe source volume 140 is designated read/write. User I/Os 105 aredirected to the source volume 140.

At a flow point 340, the method 300 is complete.

Alternative Embodiments

Although preferred embodiments are disclosed herein, many variations arepossible which remain within the concept, scope, and spirit of theinvention, and these variations would become clear to those skilled inthe art after perusal of this application.

1. A method for aligning a target volume and a source volume after saidsource volume has been written to, including comparing information abouta set of snapshots associated with said source volume with informationabout a set of snapshots associated with said target volume to determinecommon information that said set of snapshots associated with saidtarget volume and said set of snapshot associated with said sourcevolume share in common; reverting said target volume back to a stateassociated with said common information; generating a new snapshot ofthe source volume; calculating a difference between a first union ofdata blocks in the snapshots associated with the source volume and asecond union of data blocks in the snapshots associated with the targetvolume; and transferring a set of data blocks corresponding to thedifference from said source volume to said target volume, therebyaligning said target volume with said source volume.
 2. A method as inclaim 1, wherein said step of comparing includes reading a set of datablocks included in said set of snapshots associated with said sourcevolume; and reading a set of data blocks included in said set ofsnapshots associated with said target volume.
 3. A method as in claim 1,wherein said step of transferring includes writing data associated withsaid source volume to said target volume.
 4. A method as in claim 1,also including saving said information about said set of snapshotsassociated with said target volume, wherein said information about saidset of snapshots associated with said target volume concerns an initialstate of said target volume prior to said step of transferring.
 5. Amethod as in claim 1, wherein said common information includes a mostrecent information that said set of snapshots associated with saidtarget volume and said set of snapshots associated with said sourcevolume share in common.
 6. A method as in claim 5, wherein said state towhich said target volume is reverted back is a state associated withsaid most recent information.
 7. A method as in claim 1, wherein saidsource volume may receive reads and writes from a client during saidstep of comparing.
 8. A method as in claim 1, wherein said step ofcomparing is performed by a set of logic coupled to both said sourcevolume and said target volume.
 9. A method as in claim 1 also includinggenerating an alarm to inform a user that data associated with saidtarget volume may be lost.
 10. A method as in claim 1, further includingremoving a set of data blocks from said target volume based upon asecond result associated with said comparison.
 11. An apparatus,including a processor, a memory and a set of instructions for aligning atarget volume and a source volume after said source volume has beenwritten to, including an instruction to compare information about a setof snapshots associated with said source volume with information about aset of snapshots associated with said target volume to determine commoninformation that said set of snapshots associated with said targetvolume and said set of snapshots associated with said source volumeshare in common; an instruction to revert said target volume back to astate associated with said common information; an instruction togenerate a new snapshot of the source volume; an instruction tocalculate a difference between a first union of data blocks in thesnapshots associated with the source volume and a second union of datablocks in the snapshots associated with the target volume; and aninstruction to transfer a set of data blocks corresponding to thedifference from said source volume to said target volume.
 12. Anapparatus as in claim 11, wherein said set of instructions includes aninstruction to save said information about said set of snapshotsassociated with said target volume, wherein said information about saidset of snapshots associated with said target volume concerns an initialstate of said target volume.
 13. An apparatus as in claim 11, whereinsaid common information includes a most recent information that said setof snapshots associated with said target volume and said set of snapshotassociated with said source volume share in common.
 14. An apparatus asin claim 13, wherein said state to which said target volume is revertedback is a state associated with said most recent information that saidset of snapshots associated with said target volume and said set ofsnapshots associated with said source volume share in common.
 15. Anapparatus as in claim 11, wherein said set of instructions includes aninstruction to generate an alarm to inform a user that data associatedwith said target volume may be lost.
 16. A method of aligning a firstvolume of data storage and a second volume of data storage, the methodcomprising: identifying a set of snapshots of the first volume;identifying a set of snapshots of the second volume; selecting a mostrecent snapshot common to the first and second volumes; reverting thesecond volume to a state associated with the most recent snapshot commonto the first and second volumes; generating a new snapshot of the firstvolume; calculating a first difference between a union of data blocks inthe snapshots associated with the first volume and a union of datablocks in the snapshots associated with the second volume; andtransferring a set of data blocks corresponding to the first differencefrom the first volume to the second volume.
 17. A method in accordancewith claim 16, further comprising: preventing users from writing to thefirst volume; and preventing users from writing to the second volume.18. A method in accordance with claim 17, further comprising thefollowing steps performed after the step of transferring a set of datablocks corresponding to the first difference: reverting the first volumeto a state associated with the most recent snapshot common to the firstand second volumes; calculating a second difference between a union ofdata blocks in the snapshots associated with the second volume and aunion of data blocks in the snapshots associated with the first volume;and transferring a set of data blocks corresponding to the seconddifference from the second volume to the first volume.
 19. A method inaccordance with claim 18, further comprising generating a new snapshotof the second volume before the step of calculating a second difference.20. A method in accordance with claim 19, further comprising removingsnapshots that are not included in the first volume from the secondvolume, the step of removing being performed before the step ofcalculating a second difference.
 21. Apparatus for aligning a firstvolume of data storage and a second volume of data storage, theapparatus comprising a memory with instructions embedded therein, and atleast one processor coupled to the memory to read and execute theinstructions, wherein the instructions, when executed by the at leastone processor, cause the at least one processor to perform the steps of:identifying a set of snapshots of the first volume; identifying a set ofsnapshots of the second volume; selecting a most recent snapshot commonto the first and second volumes; reverting the second volume to a stateassociated with the most recent snapshot common to the first and secondvolumes; generating a new snapshot of the first volume; calculating afirst difference between a union of data blocks in the snapshotsassociated with the first volume and a union of data blocks in thesnapshots associated with the second volume; and transferring a set ofdata blocks corresponding to the first difference from the first volumeto the second volume.
 22. Apparatus according to claim 21, wherein theinstructions, when executed by the at least one processor, further causethe at least one processor to (1) prevent users from writing to thefirst volume, and (2) prevent users from writing to the second volume.23. Apparatus according to claim 22, wherein the instructions, whenexecuted by the at least one processor, further cause the at least oneprocessor to generate a new snapshot of the second volume before thestep of calculating a second difference.
 24. Apparatus according toclaim 23, wherein the instructions, when executed by the at least oneprocessor, further cause the at least one processor to remove snapshotsthat are not included in the first volume from the second volume beforethe step of calculating a second difference.
 25. An article ofmanufacture comprising a memory having program code embedded therein,the program code comprising instructions that, when executed by at leastone processor coupled to a first volume of data storage and to a secondvolume of data storage, cause the at least one processor to perform thesteps of: identifying a set of snapshots of the first volume;identifying a set of snapshots of the second volume; selecting a mostrecent snapshot common to the first and second volumes; reverting thesecond volume to a state associated with the most recent snapshot commonto the first and second volumes; generating a new snapshot of the firstvolume; calculating a first difference between a union of data blocks inthe snapshots associated with the first volume and a union of datablocks in the snapshots associated with the second volume; andtransferring a set of data blocks corresponding to the first differencefrom the first volume to the second volume.
 26. An article ofmanufacture according to claim 25, wherein the instructions, whenexecuted by the at least one processor, further cause the at least oneprocessor to (1) prevent users from writing to the first volume, and (2)prevent users from writing to the second volume.
 27. An article ofmanufacture according to claim 26, wherein the instructions, whenexecuted by the at least one processor, further cause the at least oneprocessor to generate a new snapshot of the second volume before thestep of calculating a second difference.
 28. An article of manufactureaccording to claim 27, wherein the instructions, when executed by the atleast one processor, further cause the at least one processor to removesnapshots that are not included in the first volume from the secondvolume before the step of calculating a second difference.